Repository navigation
feat: wire update, publish/unpublish and job status to real backends … - #13
Open
lirantul123 wants to merge 14 commits into
Open
lirantul123 wants to merge 14 commits into
lirantul123 wants to merge 14 commits into
Conversation
Rename boilerplate identity to 3d-ops-trigger across package.json, openapi, helm chart/values, catalog-info and README. Pin Node>=24 (engines) and wire core deps: @map-colonies/jobnik-sdk, @map-colonies/3d-shared (local file link), zod.
Define the four 3D operations in openapi3.yaml (POST /record, DELETE
/record/{id}, PATCH /record/{id}, PATCH /record/status/{id}) and wire the
record resource (router, controller, manager) with tsyringe. Managers are
stubbed pending Jobnik/catalog integration. Replace boilerplate example
resources. Integration tests at the HTTP seam + a RecordManager unit test.
Add ValidationManager that validates ingestion metadata using zod schemas from @map-colonies/3d-shared: core fields (new3DLayerMetadataSchema), footprint 2D/3D (geometrySchema), and date-before-date. Invalid metadata throws AppError(400) routed via the controller error seam. Wire the validator into RecordManager's ingestion path. Restore AppError (now used). Unit + integration tests.
Add JobnikClient wrapping the @map-colonies/jobnik-sdk Producer. - Ingestion: create 3D-Ingestion job + validation stage/task (+ data-extraction stage/task only for 3tz input). - Delete: validate deletability (record exists, productType not QuantizedMeshDTMBest, productStatus UNPUBLISHED) then create a 3D-Delete job. RecordManager ingest/delete now return real Jobnik jobs. Add externalServices.jobManager config (schema, default, custom-environment-variables, helm). Unit + integration tests.
Add ValidationManager check that rejects an ingestion when an in-flight Jobnik ingestion job already exists for the same productName, complementing the catalog unique-name check (which only sees finished records). JobnikClient.hasInFlightIngestionJob queries GET /v1/jobs by job name and filters client-side on non-terminal status + data.metadata.productName, paginating through all pages. Query failures raise AppError. JobnikClient is now a singleton so RecordManager and ValidationManager share one SDK instance instead of double-registering its metrics. Interim until Jobnik ships native duplicate support (epic MAPCO-11833).
Add tileset validation to the ingestion flow for both 3tz and folder inputs: read tileset.json, extract the model polygon from its bounding volume, and require the footprint to intersect and cover the model above a configurable threshold. TilesetReader reads tileset.json without a zip dependency: for a 3tz it reads the first local file entry directly via a file handle (real 3tz archives break yauzl's central-directory parsing), guarding against data-descriptor/ZIP64 entries whose size is absent from the local header; for a folder it reads the file from disk. Polygon extraction (region/sphere bounding volumes) lives in @map-colonies/3d-shared. Coverage is checked with turf against validation.percentageLimit (default 10), wired through schema, config, custom-environment-variables and helm. Unit and integration tests included.
Return ValidationResponse {isValid, message} from the tileset polygon
extraction and footprint-intersection checks (mirroring 3d-gateway), route
every failure through a single rejectValidation() that logs the outcome and
throws AppError so the message reaches the client. Move ValidationResponse to
common/interfaces. Match 3d-gateway's coverage/far-from-model messages and add
a coverage-below-threshold test.
…(MAPCO-11833)
Replace the stub PATCH handlers and add a job-status endpoint, making the
service a real successor to 3d-gateway + store-trigger:
- PATCH /record/{id}: validate the update (record exists, not being-deleted,
v2 partial schema, footprint, classification, unique name), reject records
that exist in extractable-management (409), convert the footprint to 2D and
patch the catalog.
- PATCH /record/status/{id}: validate the record, extractable guard, then
change status in the catalog.
- GET /jobs/{jobId}: return the Jobnik job status and completion percentage.
Add CatalogCall getRecord/patchMetadata/changeStatus, an ExtractableCall client
(externalServices.extractable), JobnikClient.getJobStatus, and a job module
(model/controller/route mounted at /jobs). Unit and integration tests included.
…us and bundle (MAPCO-11833)
Bring the service closer to the 3d-gateway behaviour:
- POST /record/validate: run the full ingestion validation and return
{isValid, message} without creating a job.
- Ingestion validation also enforces modelPath under the configured base path
(validation.basePath) and productId uniqueness in the catalog.
- Ingestion converts the footprint to 2D before creating the job.
- Delete now sets the catalog record status to BEING_DELETED after the delete
job is created.
- Add an openapi:bundle script (redocly) producing bundledApi.yaml.
Skipped by design (Jobnik model, not store-trigger): PV-path rewriting,
pre-minted uuid modelId, StoreTriggerCall.
…833)
- GET /record/canDelete/{id}: dry-run delete validation returning
{isValid, message}, matching 3d-gateway's GET /models/canDelete/{recordId}.
- Rename the bundle script to bundleOpenApi and commit bundledApi.yaml
(tracked, like 3d-gateway and store-trigger; removed from .gitignore).
…a groups (MAPCO-11833)
Restructure the routes to mirror 3d-gateway + store-trigger groupings instead of
a single /record tag:
- jobOperations: POST /jobOperations/ingestion, POST /jobOperations/delete
- jobStatus: GET /jobStatus/{jobId}
- models: POST /models/validate, GET /models/canDelete/{recordId}
- metadata: PATCH /metadata/{identifier}, PATCH /metadata/status/{identifier}
Delete is now POST with a body ({id}) like store-trigger. Each group has its own
controller and router; RecordManager and JobManager remain the shared business
models. OpenAPI, generated types, bundledApi and integration tests updated.
…-11833)
Mirror 3d-gateway's models group (renamed to our domain term "record"): all
model operations live under one records tag, metadata and jobStatus stay
separate.
- records: POST /records, POST /records/validate, GET /records/canDelete/{recordId},
DELETE /records/{recordId}
- metadata: PATCH /metadata/{identifier}, PATCH /metadata/status/{identifier}
- jobStatus: GET /jobStatus/{jobId}
Delete reverts to a REST DELETE. Merge the jobOperations and models modules into
a single records module (recordManager moved under it); update routers,
serverBuilder, containerConfig, OpenAPI, bundledApi and tests.
…CO-11833) Mint a modelId (uuid) at ingestion time, pass it into the Jobnik job data so the finalizer has a stable catalog id, and tag the trace span with it. Wrap the create flow in a try/catch that passes AppError through and maps unexpected errors to a 500, mirroring 3d-gateway's createModel. productSource and PV-path rewriting are intentionally omitted: productSource does not exist in the v2 metadata schema, and path rewriting is store-trigger mount-specific (revisit only if the Jobnik worker contract requires it).
- move record/job DTO aliases into @common/interfaces, dropping the duplicate UpdatePayload and the local JobStatusResponse - consume RecordStatus/ProductType from @map-colonies/types instead of hardcoded status/productType literals (adds @map-colonies/types + mime-types) - split ValidationManager into IngestionValidator, MetadataValidator and TilesetValidator, with shared errors.ts and a classification helper - dedupe CatalogCall error handling behind a single request helper
lirantul123
force-pushed
the
tileset-extraction
branch
from
November 1, 2026 13:27
69d89f2 to
b7243a3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…(MAPCO-11833)
Replace the stub PATCH handlers and add a job-status endpoint, making the service a real successor to 3d-gateway + store-trigger:
Add CatalogCall getRecord/patchMetadata/changeStatus, an ExtractableCall client (externalServices.extractable), JobnikClient.getJobStatus, and a job module (model/controller/route mounted at /jobs). Unit and integration tests included.